A mixed reality (MR) system provides, to the user, a mixed reality space image obtained by combining a physical space image and a virtual space image generated in correspondence with the viewpoint position, orientation, and the like of the user. The MR system can present to the observer as if a virtual object were existing on a physical space, and allows the observer to make an observation with a sense of actual dimensions and higher reality than a conventional virtual reality (VR) system.
On the other hand, designs (shapes and designs) using three-dimensional (3D) CAD have become mainstream in the design and manufacture fields. In this case, as a method of evaluating an object designed by the 3D-CAD, a method which displays data (solid format) created by the 3D-CAD on the screen of a computer as a 3D-CG and visually evaluates it, a method that produces a simple prototype (simple mockup) by a rapid prototyping apparatus or the like, and tactually evaluates it, and the like are popularly used.
However, with the method that displays 3D-CAD data on the screen of a computer as a 3D-CG, evaluation is made on the virtual space, and the object cannot be evaluated to have a sense of actual dimensions on the physical space. Also, the method that produces a simple prototype (simple mockup) by a rapid prototyping apparatus or the like is effective to recognize a rough shape, and the like, but this method cannot reproduce detailed information designed on the 3D-CAD such as details, colors, and the like of a design due to limitations on machining accuracy, materials, and so forth.
In order to evaluate design data in a situation closer to a completed product, the following method is proposed. That is, 3D-CG data created by converting 3D-CAD data is superimposed on a simple prototype (simple mockup) produced based on the 3D-CAD data by a rapid prototyping apparatus or the like using the MR system while matching the position and orientation. As a result, visual and tactual evaluations are realized at the same time, and the design data can be evaluated in a state closer to a completed product.
Furthermore, a method of arranging indices on a simple prototype, detecting the arranged indices from image information of the physical space input as image data, and correcting registration between the physical space and virtual space using the detected position and orientation information of the indices has also been proposed.
However, the conventional method can only handle a simple prototype which is a rigid body. More specifically, when a simple prototype has a movable part, it is impossible to superimpose the 3D-CG data on the simple prototype while moving the 3D-CG data in correspondence with the movement of the movable part. Also, it is impossible to use the position and orientation information of the indices arranged on the movable part so as to correct registration between the physical space and virtual space.